In [1]:

The aim of this script

We want to determine the location of buoys using a series of gps labeled photos.

This allows us to place cameras on ships that can automatically alarm us, if anything were to be missing.

Data exploration

In the folder db/test we find three photos. The csv file db/test_location gives us the gps locations of the photos. We display the data herunder.

In [99]:
In [2]:
     lat    lon
1 137007 425083
2 137010 425082
3 137013 425080
4 137016 425077
5 137018 425079

The first task, and the task in which we preform some deeplearning, is to determine which pixels in the photo belong to the buoy and which pixels do not.

For this we train an network to classify the pixels in the image. In the folder db/boeien we have a large number of photos of buoys and in train_labels.csv we have for each photo a square indicating the position of the buoy.

Let's have a look at the positions table

In [3]:
X.1Xnaamxminyminxmaxymaxwxwy
1 1 db/boeien/Frame000012.jpg 8 571 51 636 43 65
2 2 db/boeien/Frame000013.jpg 82 576 123 630 41 54
8 8 db/boeien/Frame000019.jpg526 580 567 636 41 56
9 9 db/boeien/Frame000020.jpg609 579 644 630 35 51
10 10 db/boeien/Frame000021.jpg684 579 726 637 42 58
11 11 db/boeien/Frame000022.jpg759 580 805 641 46 61

Let's display the image of a random row i of the table and let's draw a square according to the xmax, ymax, xmin and ymin

In [4]:

The data seems to be allright.

Per pixel classification

Now lets built a network to reproduce such an image segmentation.

In [5]:

Now let's train this network.

In [6]:
[1] "db/boeien/Frame000274.jpg"
[1] "db/boeien/Frame002869.jpg"
[1] "db/boeien/Frame000022.jpg"
[1] "db/boeien/Frame002661.jpg"
[1] "db/boeien/Frame003800.jpg"
[1] "db/boeien/Frame003802.jpg"
[1] "db/boeien/Frame000020.jpg"
[1] "db/boeien/Frame003802.jpg"
[1] "db/boeien/Frame003997.jpg"
[1] "db/boeien/Frame003086.jpg"
[1] "db/boeien/Frame003809.jpg"
Error in py_call_impl(callable, dots$args, dots$keywords): TypeError: 'NoneType' object is not callable

Detailed traceback: 
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2042, in run
    _run_using_default_session(self, feed_dict, self.graph, session)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 4490, in _run_using_default_session
    session.run(operation, feed_dict)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1323, in _do_call
    return fn(*args)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
    status, run_metadata)

Traceback:

1. train_step$run(feed_dict = dict(x = im, label = lab))   # at line 17 of file <text>
2. py_call_impl(callable, dots$args, dots$keywords)

Let's see how the network preforms on our test data.

In [ ]:

If evrything went well the result should look a bit like the following images.

Determining the position in the photo

In the previous step we obtained a heatmap in which pixels are brither when the computer deems it likely that the pixel is part of the buoy.

If the sum of all pixels around a pixel cross a certain threshold we say it is part of a buoy. Next we downsize the photo a factor 10 to merge all pi

Let's

In [ ]:

Calculating the angle with the camera

Based on the position of the buoy in the photo we can calculate the angle the buoy makes with the camera.

The distance between the film and the lens was 4 cm and the width of the film was 5 cm.

In [ ]:

Calculating the lines on which the buoys lie

Based on the locations table constructed in the above we can calculate the position of the buoy.

First we calculate the angle the boat makes with respect to the coordinate system. We add this value to the angle between the buoy and the camera with respect to the coordinate system. We add 0.25 * pi since the camera films perpendicular to the ships movement.

In [ ]:

Based on this we can calculate the coefficiet and offset of the line with respect to the coordinate system.

In [ ]:

Calculating the position of the buoy

The buoy lies on the intersection of these lines. We calculate the intersection of the first two photos.

In [ ]:

Plot positions on a leaflet

We calculate the position in wgs coordinates and plot it in a leaflet.

In [ ]:
In [ ]:
In [ ]: